There is a constant demand for increased data availability and bandwidth in computer networks. This has driven the development of higher data transmission rates on network links, such as Ethernet links, from the order of tens of megabits per second to hundreds of gigabits per second. However, the migration to these new technologies may be expensive and disruptive to the operation of data centers and users that employ them. An alternative method to increase bandwidth is to logically group multiple network interface ports together so that they appear to clients as a single network interface port, commonly referred to as a virtual network interface port. This practice is commonly referred to as simply teaming, network interface controller (NIC) teaming, link aggregation, link bundling, port trunking and NIC bonding, and is generally referred to herein as network interface adapter teaming, adapter teaming or simply teaming. Another advantage of NIC teaming is increased data availability by supporting failover to a different port of the team if one of the ports fails.
The most widely used suite of networking protocols is the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols. Generally speaking, a TCP segment includes a header and payload that are encapsulated within an IP packet. The IP packet includes a header and payload that are encapsulated within a data link layer packet, such as an Ethernet frame, or packet. Historically, the Ethernet frame is received by the network interface adapter and transferred to the system memory of the host computer, which is a server system in many cases. The network protocol stack of the host operating system, which is executed by the host CPU, then processes the frame according to the various protocol layers. The processing involves: examining various fields of the packet headers; performing error checking by generating checksums of the packet words and comparing the generated checksum with the checksum included in the packet; generating and transmitting acknowledgement packets on the network to acknowledge receipt of the received packets; and establishing connections with remote hosts, which involves processing packets and generating and transmitting packets; among others. For outgoing packets, the host CPU generates the checksums and headers, which includes calculating sequence numbers performing sliding window calculations to accomplish network congestion control. The CPU generates the packets in the system memory, the packets are transferred to the network interface adapter, and the adapter transfers the packet from system memory out of its ports onto the network.
It was found that a significant amount of the CPU bandwidth was being consumed by processing the TCP/IP packets and in some cases causing poor performance of applications running on the host. A solution developed to address this problem was to offload some the TCP/IP processing to the network interface adapter. The portion of the network adapter devoted to this task is commonly referred to as a TCP Offload Engine (TOE). A TOE stores state of a TCP session to enable it to offload the system CPU by performing some or all of the TCP/IP processing. TOEs may also reduce the amount of traffic on the host local bus, such as the Peripheral Component Interconnect (PCI) bus, by alleviating the need for some packets and/or headers to travel from system memory across the PCI bus to the network adapter, such as the acknowledgement packets, which are generated by the TOE which is on the network interface side of the PCI bus rather than the system memory side of the PCI bus.
Unfortunately, the performance benefits enjoyed by separately employing network adapter teaming and separately employing TOEs have been difficult to realize when using the two in combination. This appears to be primarily due to the fact that the network switch to which the individual host network interface adapters are linked may choose to transmit the packets of a TCP session to a different physical port of the virtual adapter than the physical port of the virtual adapter that the host chooses to transmit the packets of the TCP session. That is, the output port selection algorithms (commonly referred to as hashing algorithms) of the network switch and the host may be different. This is problematic because the TOE requires the TCP session state in order to perform its protocol offload functions. This causes the copying of packets within system memory and/or the different network adapters that are receiving the packets and transmitting the packets for the TCP session, which results in wasted system memory, local I/O bus and/or CPU bandwidth and in some cases overall system performance that is worse than obtained without the TOE services. For this reason, major network solution providers have chosen not to support TOE in combination with predominate teaming protocols, such as LACP and IEEE 802.3ad.
Therefore, a solution is needed to improve the performance of a host computer system that combines network adapter teaming and TOEs.